Best Practices in Testing and Reporting Performance of Biometric Devices

Extracts from this report may be reproduced provided the source is acknowledged and the extract is not taken out of context. SUMMARY The purpose of this document is to summarise the current understanding by the biometrics community of the best scientific practices for conducting technical performance testing toward the end of field performance estimation. Our aims are: • To provide a framework for developing and fully describing test protocols. • To help avoid systematic bias due to incorrect data collection or analytic procedures in evaluations. • To help testers achieve the best possible estimate of field performance while expending the minimum of effort in conducting their evaluation; • To improve understanding of the limits of applicability of test results and test methods. This document is a revision of the original version " Best Practices in Testing and Reporting Performance of Biometric Devices – Issue 1 " , released in February 2000. This new issue is informed by several important developments in the intervening two years: • We have had extensive comments on the original document from the biometric community. We are greatly indebted to all of those who commented on Issue 1. Even this new issue remains a living document, open to review and correction by the biometrics community. We encourage written comments and criticism on all aspects of the organisation and content. • We conducted and completed the CESG/NPL Biometric Test Programme of six biometric technologies and seven vendor products, learning much about the shortfalls of the Issue 1 document. • We have also noted the protocols and results of other recent test programmes. • We have made some headway in understanding the statistical relationship between test size and confidence intervals. Issue 1 drew heavily on " The Speaker Verification Test Protocol " and " An Introduction to Testing Biometric Systems " , both produced by NIST. This Issue 2 is intended to be fully compatible with those documents, while integrating their salient points more clearly and more seamlessly than in Issue 1.